EE264 Project : Simultaneous Structure and Texture Image Inpainting

Image inpainting is an old technique for filling in missing or damaged regions of an image. The general problem is: given an image and a user-defined region of the image to be filled, fill the region in such a manner that one cannot tell that the region has been altered. In this project I have implemented the method proposed by Bertalmio et al in their 2003 paper “Simultaneous Structure and Texture Image Inpainting” to accomplish this task automatically on digital images. 1 Traditional Image Inpainting Image inpainting is a method of modifying images in such a manner that one cannot detect the modification to the image. A typical application of inpainting is the restoration of old and decaying paintings. In this scenario, cracks or other defects have appeared in the work and one wishes to restore the piece so that there are no visible defects. An example of such restoration can be seen in Figure 1. To perform this restoration one has to paint over the cracks in order to hide their existence. If a photograph or some other copy of the original, undamaged artwork exists that can be used as a reference it is a relatively straightforward, if not tedious, task to painstakingly recreate the original image by using the reference image. However, it is often the case that a reference image is unavailable. In this case it is up to the restorator to fill in the damaged regions on their own. A common approach for accomplishing this inpainting task is to fill in the damaged regions using information from the surrounding areas. This includes extending lines and curves entering the damaged regions, as well as filling in the regions with the color and texture information from the surrounding area. A second example of when image inpainting is needed is when one wishes to remove an undesired object or individual from an image. A classic example of such a scenario are the images which the dictator Joseph Stalin had altered in order to remove political enemies from photographs, Figure 1: On the left is the damaged image. On the right is the restored version. Note that cracks in the image have been believably filled in. Image courtesy of www.topcstudio.com. Figure 2: On the left is the original photograph, on the right is the same photograph with Nikolai Yezhov removed. Image obtained from http://www.newseum.org/berlinwall/ commissar_vanishes/index.htm.